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PREFACE 

Part  of  the  Project  RAND  research  program  consists 
of  basic  supporting  studies  in  mathematics.  The  mathe¬ 
matical  research  presented  here  deals  with  an  inverse 
problem  in  automatic  control  theory. 
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SUMMARY 


The  authors  investigate  the  inverse  of  a  basic 
problem  of  automatic  control  theory,  namely,  being 
given  an  optimal  policy  and  the  equations  and  inequali¬ 
ties  representing  the  constraints  of  the  problem,  they 
seek  the  criterion  function  or  functions  that  lead  to 
this  policy;  methods  of  solution  are  found  in  some 
simple  cases. 
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AN  INVERSE  PROBLEM  IN  DYNAMIC  PROGRAMMING 
AND  AUTOMATIC  CONTROL 


1.  INTRODUCTION 

One  of  the  basic  problems  of  modem  control  theory 
Is  that  of  minimizing  a  functional 

r  T 

(1.1)  J(x,y)  -  J  g(x,y)dt, 

0 

where  x  and  y  are  N-dlmensional  vector  functions 
subject  to  the  relations 

(1.2)  (a)  x(0)  -  c , 

( b )  g|«h(x,y), 

(c)  r^x.y)  £  0,  1  »  1,2,.. ,,k. 

Problems  of  this  nature  may  be  attacked  by  means  of 
classical  variational  techniques  and  extensions  € l] ,  [2], 
or  by  means  of  dynamic  programming.  The  latter  emphasizes 
the  physical  concept  of  feedback  control,  since  it 
attempts  to  determine  directly  the  optimal  policy 
v  ■■  v(c,T)  as  a  solution  of  the  equation 

(1.3)  fm  »  max  t g( c , v)  +  (grad  f,h)l, 

v 

where  . r^(c,v)  £  0,  i  =  l,2,...,k,  and 

(1.4)  f(c,T)  «  min  J(x,y). 


-2- 


In  this  Memorandum  we  initiate  investigation  of  an 
inverse  problem  of  some  importance:  "Given  the  optimal 
policy  v  =  v(c,T)  and  the  descriptive  equations  of 
(1.2),  what  criterion  function  possesses  this  optimal 
policy?"  The  study  of  inverse  problems  forms  a  chapter 
in  the  calculus  of  variations,  with  emphasis  laid  on 
determining  Lagranglans  having  a  given  family  of  curves 
as  Euler  arcs.  (See  [4],  [5}.) 

In  the  simple  case  where  no  constraints  are  present 
and  h(x,y)  s  y,  we  obtain  a  simple  linear  partial 
differential  equation  which,  in  principle,  determines  f 
and  g.  In  some  cases,  this  leads  to  an  effective 
analytic  solution. 

2.  SCALAR  CASE 

Consider  the  problem  of  minimizing 


(2.1) 


fT 

J  =J  g(u,u’  )dt, 
0 


with  u(o)  =  c,  where  u(t)  is  a  scalar  function. 
Writing  f(c,T)  for  the  minimum  value,  we  obtain  in  the 
usual  fashion  [  3)  the  equation 


(2.2)  fT  -  min  tg(c,v)  +  vfc)  . 


nils  in  turn  leads  to  the  two  equations 
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(2.3)  fT  -  g(c,v)  +  vfc, 

0  =  gv,+  fc, 
or 

(2.4)  f T  =  g  —  vgv, 

f  »  —  2  . 

C  6V 

Eliminating  f  by  partial  differentiation,  we  have  the 
equation 

(2.5)  (g  -  vgv)Q  *  -  (gv)T, 

which,  in  principle,  determines  g,  given  the  function 
v(c,T). 

This  equation  in  g  involves  second  partial 
derivatives  with  respect  to  v  and  is  not  immediately 
tractable. 

3.  THE  FIRST-ORDER  EQUATION  FOR  fT 

Returning  to  (2.3),  let  us  proceed  in  the  following 
fashion.  Write 

(3-1)  (fT  -  vfc)T  -  g(c,v)T  -  gyvT  «  -  Vipf c, 

upon  taking  account  of  the  second  equation  of  (2.3). 
Hence, 


(3.2) 


fTT  ~  Vc  “  vfcT 


I 


or 

(3.3)  (fT)T  -  v(fT)c  -  0, 

a  first-order  linear  equation  for  fT,  given  the 
function  v  **  v(c,T),  Once  f  is  determined,  we 
obtain  g  from  (3-3). 

As  is  iriunediately  seen,  the  determination  of  fT 
is  quite  simple  if  we  ask  for  a  criterion  function  which 
yields  an  optimal  policy  of  the  form  v(c,T)  «  ^(c)k(T). 

4.  MULTIDIMENSIONAL  CASE 

Consider  the  multidimensional  problem.  The  basic 
equation  is 

(4.1)  fT  =  min  tg(c,v)  +  (grad  f,v)l . 

v 

In  terms  of  the  components,  we  obtain  the  equations 

N 

(4.2)  fT  =  g(c , v)  +  2  f  v., 

1  i=l  ci  1 


which  yield,  in  exactly  the  same  fashion  as  before,  the 
equation 


(  f  j)rp 


N 

2 


(fT)c  1 

i-1  1  Ci 


i* 


(4.3) 


Again,  the  determination  of  fT,  given  that 
v(c,T)  »  q>(c)k(T)  (where  k(T)  is  a  scalar  and  <p(c) 
is  a  vector),  is  reduced  to  a  conventional  problem  in 
first— order  partial  differential  equations  which  can  be 
treated  by  means  of  characteristics  or  otherwise. 

5.  DISCUSSION 

Finally,  let  us  note  that  if  one  of  the  significant 
functions  of  a  variational  problem  is  given,  it  is 
frequently  possible  to  form  equations  for  the  others  in 
terms  of  it.  For  example,  if  the  integrand  g(c,v) 
is  given,  then  it  is  possible  to  find  a  first-order 
partial  differential  equation  for  the  optimal  control 
function  v(c,T).  In  fact,  upon  expanding,  we  see  that 
equation  (2.5)  yields  the  desired  relationship.  These 
observations  have  Interesting  consequences  in  analytical 
dynamics  and  control  theory  which  will  be  discussed 
elsewhere. 
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